This invention relates to a computer system for data manipulation and in particular, but not exclusively, to a computer system arranged to manipulate data to ascertain future trends. Related methods are also provided.
Linear, and more generally mathematical, programming optimization techniques have been in steady commercial use on the computing systems of the day since the mid 1950s. Sometime in the mid 1970s, independently at both the US Energy Department and the World Bank, the concept of a modelling language for mathematical programming arose.
Representation of a model in such a language is to the numerical optimization of a mathematical program the analogue of what a database management system schema is to a flat file. It is straightforward for the analyst to turn a mathematical description of a model into statements in a modelling language which then communicates directly with the solver—optimization routine.
In the intervening years a variety of sophisticated modelling languages have come into every day use for the creation, solution and modification of large scale deterministic linear, nonlinear and discrete optimization models of practical planning problems.
Dynamic Stochastic Programming (DSP) is a numerical method for solving planning problems that involve uncertainty, now and in the future. Generally, the method involves writing the problem in a mathematical programming language, and passing that program to a suitable solver. However, solving such problems is computationally intensive, making it nigh impossible to solve realistic problems in acceptable time. The present difficulty of the method stems from a number of issues with current DSP tools. It has therefore been a problem to solve such problems involving uncertainty, now and in the future, in an acceptable time or at all. There is a desire to solve such problems in order that predictions about the future can be made. Such predictions are useful for a wide variety of fields including: the supply and demand for energy, oil, or telecommunications bandwidth or in financial markets. Future predictions about the requirement for bandwidth, energy, oil and the like may be useful in determining the size of plant to construct, etc.
There is an issue of how to represent uncertain future states of the world. Generally this is done by simulation: forecasting alternative scenarios for the future. Stochastic programming involves combining all of these alternative scenarios into the Deterministic Equivalent Problem (DEP). Here lies the problem: too few scenarios and the solution is not robust, too many and the problem cannot be solved in reasonable time. This is particularly problematic for DSP in which there are multiple time-steps and the number of scenarios grows geometrically.
The second issue is the choice of a suitable solver: one that is adapted to the nature of the problem. General-purpose solvers attempt to map the entire DEP into memory, which constrains the practitioner to unrealistically simple DSP's or large amounts of memory and computing power would be required to solve DEP of more complexity. The problem thus arises as to how a realistic problem can be solved in reasonable time with the computing power that it is currently readily available.